1. Field of the Invention
The present invention broadly relates to data processing systems. The invention has particular application in a multitasking pen-based computer system in which a single task is dedicated to routing user input such as keyboard, mouse or stylus events to various computer programs executing concurrently under user control.
2. Description of the Prior Art
Pen software extensions to operating systems, such as the preferred IBM OS/2 operating system, permit users to write directly into a window characters that are subsequently sent to a gesture and handwriting recognition engine for recognition. By default, the system treats the hand printed shapes as gestures unless the window is one which accepts text such as an edit field or a text entry window. Pen-based systems are described generally in U.S. Pat. No. 5,252,951 by Alan Tannenbaum, et al. entitled "Graphical User Interface with Gesture Recognition in a Multi-Application Environment".
By convention, fields that accept text indicate so by displaying an I-beam mouse pointer when the pointer is positioned over the text field. The operating system provides a standard I-beam mouse pointer for applications to use so that, 1) all entry fields have a uniform look and, 2) the developer of each application does not have to design and ship a custom version of an I-beam mouse pointer with its respective product.
The pen-based system software takes advantage of this standard I-beam to know the user is writing into a text field. As the user moves the stylus close to the surface and begins to write, the system moves the mouse pointer to the location of the stylus. If the mouse pointer is in a text field, the pointer becomes an I-beam. The pen-based system is unaware of the pointer shape but compares the ID of the pointer to the ID of the system supplied I-beam pointer. If the IDs match, the system takes two actions. First, the system changes the mouse pointer from an I-beam to a pen figure to alert the user that he or she may write directly into the field with the stylus. Second, the system treats the handwritten characters primarily as text rather than gestures. The examination of application, window, and pointer characteristics for the purpose interpreting pen or stylus input is discussed in IBM Technical Disclosure Bulletin, Vol. 38, No. 09, September 1995, entitled, "Data Interpretation Techniques for a Pen-based Computer".
The problem pen faces however is that many application developers ship their applications with their own version of an I-beam pointer. These developers design their own I-beam pointer shapes for aesthetic reasons or simply to distinguish their applications from those of the competition. In all cases, though, these custom I-beams still retain some resemblance to the standard I-beam shape to ensure good human factors. Otherwise, users might become confused and not perceive that the fields accept text input. Unfortunately, most of the dominant word processor applications are shipped with their own I-beam pointers rendering the presently existing pen-based systems techniques of comparing the pointer IDs somewhat ineffective.
Another problem in pen-based systems is ascertaining when an application is unable to process mouse and keyboard input. An application normally signals this `busy` condition by displaying an hour glass pointer. This is usually sufficient indication and the user ceases input until the hour glass pointer gets restored to the arrow pointer. In many cases however, while the application is in a busy state, there is already mouse or keyboard input in the system queue that has been directed to other active applications on the desktop. But given the design of both the OS/2 and Windows operating systems, the enqueued input cannot be dequeued and dispatched to the target application until the busy application has first, in effect, told the operating system it has finished dequeueing mouse and keyboard input. The operating system has no way of knowing the application is momentarily in a busy state, although the user does. If the operating system had way of detecting this busy state, it could summarily examine the input queue and dequeue/route any mouse or keyboard input that was directed at a non-busy application that was waiting on user input. The result would be the removal of a processing bottleneck.
It is therefore an object of the invention to optically recognize mouse pointers displayed by various computer program applications to determine the dynamic state or environment of the applications.
It is another object of the invention to perform accurate pointer shape recognition, ensuring that non-target pointer shapes are rejected with a high degree of confidence.
It is still a further object of the invention to perform pointer shape recognition in a computationally efficient manner so that the overhead added by the recognition process has no discernible effect on the responsiveness or performance of the computer.
It is yet a further object of the invention to perform pointer shape recognition on computers with varying display resolutions and color capabilities.